Can Boosted Randomness Mimic Learning Algorithms of Geometric Nature? Example of a Simple Algorithm That Converges in Probability to Hard-Margin SVM

In the light of general question posed in title, we write down a very simple randomized learning algorithm, based on boosting, that can be seen as nonstationary Markov random process. Surprisingly, decision hyperplanes resulting from this algorithm converge probability to exact hard-margin solutions support vector machines (SVMs). This fact is curious because hyperplane not statistical solution, but purely geometric one-driven by margin maximization and strictly dependent particular locations some data points are placed contact region two classes, namely vectors. The proposed detects vectors probabilistically, without being aware their definition. We give proofs main convergence theorem several auxiliary lemmas. analysis sheds new relation between boosting SVMs also nature SVM since they now regarded equivalently limits certain trajectories. experimental part, correctness verified against known solvers: libsvm, liblinear, optimization packages: cvxopt (Python) Wolfram Mathematica.